The present invention relates to creation and viewing of three-dimensional moving or still images, and in particular to xe2x80x9cautostereoscopicxe2x80x9d systems that do not require special wearable peripheral devices.
For well over a century, researchers have proposed and developed a variety of devices that allow viewing of images or graphic designs in three dimensions. The concept of true xe2x80x9cstereoscopicxe2x80x9d viewing holds tremendous potential for applications ranging from entertainment to scientific visualization and basic research. Stereoscopically portrayed stories and events carry the impact and immediacy of true realism, allowing viewers to be drawn into an experience with the sense that they are truly there.
Efforts to date have largely been confined to specific, controlled environments. For example, for decades movie-goers have donned special glasses to view stereoscopic films; for this application, the restricted presentation atmosphere and a captive audience willing to tolerate special viewing aids facilitated use of relatively unsophisticated optical arrangements. Similar audience receptivity underlies the current generation of commercial xe2x80x9cvirtual realityxe2x80x9d devices, which require the user to wear full-vision headgear that imparts an xe2x80x9cimmersivexe2x80x9d three-dimensional environment.
Displays involving eyeglasses or headgear control what enters the eyes rather than what exits the display. The wearable apparatus covering the user""s eyes allows separate information to be provided to each eye. The left-eye and right-eye images differ in perspective but not content, so the viewer integrates the images into a single, stereoscopic picture. Early three-dimensional film systems displayed left-eye and right-eye images in separate colors, which were directed to the appropriate eye by special glasses having lenses tinted with one or the other of these colors. More recent adaptations of this approach code the left-eye and right-eye images with orthogonal polarizations, and utilize eyeglasses with orthogonally oriented polarizers.
The disadvantages to systems that require eyeglasses or wearable headgear are numerous and well-known. Beyond the inconvenience and unnaturalness of wearing an appliance, the user may also experience headaches or eye strain. Head-mounted displays suffer the additional disadvantage of being single-user devices, isolating viewers from one another and preventing them from sharing the three-dimensional experience with others.
Another popular form of stereoscopic display relies on lenticular optical technology, which utilizes a linear array of narrow cylindrical lenses to create separate spatial viewing zones for each eye. Image information for the different view zones is spatially separated in the back focal plane of the cylindrical lenslets, allowing the lenslets to direct this information only to a narrow area of the viewing plane. Recent adaptations of this approach utilize liquid crystal display (LCD) panels or LCD-projected images to provide an updatable display medium for creating the spatially separated information behind the cylindrical lenses. Lenticular displays also suffer from certain drawbacks, however, such as poor image resolution (due both to the need to divide the overall resolution of the single image-producing device over a plurality of view zones, and to diffraction). Lenticular designs are also difficult to adapt to multi-user environments.
Other approaches to stereoscopic image presentation include so-called xe2x80x9cvolumetricxe2x80x9d displays (which utilize a medium to fill or scan through a three-dimensional space, small volumes of which are individually addressed and illuminated), and electronic holography displays. Both of these types of display require rapid processing of enormous quantities of data, even for lower resolution images, and both have significant obstacles to overcome when the displays are scaled up to accommodate larger image sizes. In addition, the volumetric displays produce transparent images which, while suitable for applications (such as air-traffic control or scientific visualization) where the illusion of solidity is less important than a wide viewing zone, do not typically provide a fully convincing experience of three-dimensionality.
Macro-optic display systems utilize large-scale optics and mirrors, as opposed to lenticular displays, to deliver each image of a stereopair to a different viewing zone. A system designed by Hattori et al. (see Hattori et al., xe2x80x9cStereoscopic Liquid Crystal Display,xe2x80x9d Proc. Telecom. Advance. Org. (TAO) 1st Int""l. Symp. (1993)) utilizes two LCDs, one providing left-eye information and the other providing right-eye information. The outputs of both LCDs are combined by a beamsplitter, with the light passing through each LCD being focused to a separate viewing zone. The Hattori et al. system utilizes a monochrome cathode-ray tube (CRT) monitor behind each LCD as the illuminating light source. Each monochrome monitor is driven by a To camera that records the viewing area in front of the display, capturing a picture of the viewer. A pair of infrared (IR) illuminators, each emitting at a different wavelength, are angled toward the viewer from different sides. Each recording camera is equipped with a bandpass filter tuned to the emitting frequency of one or the other IR illuminator. Because the illuminators each face the viewer at an angle, one of the cameras records an image of the left side of the viewer""s face, while the other records an image of the right side. A fresnel lens near each LCD projects the left-side or right-side image of the viewer""s face, as appropriate, onto the corresponding side of the viewer""s actual face. As a result, the image information from each LCD reaches only the appropriate left or right eye; for example, because light passing through the left-eye image LCD goes only to the left side of the viewer""s face, the viewer""s left eye sees only left-eye information. As the viewer moves within the viewing space, the image on the monitors moves with him, so the view zones for the left-eye and right-eye images remain properly positioned over the viewer""s left and right eyes.
This type of display offers a number of advantages over prior designs. It is xe2x80x9cautostereoscopic,xe2x80x9d so that the user receives a three-dimensional image without special wearable peripheral devices. It is capable of delivering three-dimensional images to a moving viewer anywhere within the viewing area, and can accommodate several viewers at once. In addition, because the system uses LCDs as the primary image source for the stereopairs, it is capable of generating strongly realistic, full-color images of naturalistic scenes (as well as displaying ordinary two-dimensional television or other information). And since only two LCDs are used, the total amount of information needed to drive the display is only twice that of a standard television or monitor.
The Hattori et al. design poses problems in terms of scalability, however. Because the viewer-tracking system is implemented with CRTs, larger displays will require proportionally larger CRTs and more powerful lenses. As a result, the display size, cost, and complexity expand dramatically with increase in the size of the stereoscopic image. Moreover, because this design requires a final beamsplitter to be placed between the display medium and the viewer, the resulting three-dimensional images give the psychological impression of being inaccessible to the viewer; this arises from the fact that stereoscopic images display the least distortion when the three-dimensional content is localized at or near the surface of the display medium, which is positioned behind the final beamsplitter. Other limitations of the design stem from shortcomings that generally affect CRT displays, such as varying image intensities.
Another macro-optical design was recently proposed by Ezra et al. (see Ezra et al., xe2x80x9cNew Autostereoscopic Display System,xe2x80x9d SPIE Proc. #2409 (1995)). Starting once again with two image LCDs combined by a beamsplitter, an additional system of fold mirrors and another beamsplitter were added to allow both LCDs to be backlit by a single light source. A lens disposed near each LCD images the single light source to the eyes of the viewer. These lenses are laterally offset by a do slight amount from the optical axis of the LCDs so that two separate images of the light source are created in the viewing area. Light passing through the left-eye LCD forms an image of the light source near the viewer""s left eye, while light passing through the right-eye LCD forms an image of the light source near the viewer""s right eye. By moving the light source behind the two LCDs, the left-eye and right-eye view zones can be moved to follow a viewer within the viewing area. To accommodate additional viewers, additional xe2x80x9cdynamic light sourcesxe2x80x9d can be added so as to create further view zones. More recently, this group proposed handling multiple viewers with a single, specialized illumination component rather than multiple individual light sources. This specialized component consists of a number of thin, vertical cold-cathode sources arranged in a one-dimensional array. See Woodgate et al., xe2x80x9cObserver Tracking Autostereoscopic 3D Display Systems,xe2x80x9d SPIE Proc. #3012A (1997).
This system shares many of the advantages of the Hattori et al. design described above, and overcomes the difficulties stemming from multiple illumination sources. Once again, however, the ability to scale the Ezra et al. system can be problematic. The two LCDs and the beamsplitter occupy a large space for a display of only modest three-dimensional size. As in the Hattori et al. system, the three-dimensional image is xe2x80x9ctrappedxe2x80x9d behind the output beamsplitter, making the images seem inaccessible. Finally, the array of cold-cathode sources have a limited on/off switching speed, creating possible lags in tracking speed.
The present invention addresses the foregoing limitations using a variety of innovative approaches to viewer tracking and image presentation. Although the various embodiments of the invention are well-suited to presentation of complementary stereoimages to each of the viewer""s eyes, it should be stressed that the invention is not limited to this use. More broadly, the invention is useful in any context requiring the directing of separate images to distinct spatial regions. For example, the images generated by the invention can be directed to different viewers (rather than the left and right eyes of a single viewer), so that viewers observing the same display device can view different images. Although the ensuing discussion is directed toward generation of stereoimages for a single viewer, it should be understood that this is for convenience of presentation only.
In a first embodiment, an improved xe2x80x9cthrough-the-beamsplitterxe2x80x9d approach (in which the viewer sees a stereoscopic image through a final beamsplitter located at the output of the display) utilizes light polarization to separate left-eye and right-eye stereoimages. Each of the complementary images is projected so as to be visible to only one of the viewer""s eyes. This may be accomplished by means of a viewer-locating system that acquires the locations of the viewer""s eyes, and a viewer-tracking system that directs each stereoimage to the proper location.
In accordance with this embodiment, a viewer-locating means acquires a facial image fragment of the viewer; as used herein, the term xe2x80x9cfacial image fragmentxe2x80x9d refers to a recognizable portion of the user""s face, and may include, or be limited to, a first one of the viewer""s eyes. For example, a camera at the front of the display may capture a picture of the viewer with one side of his face illuminated by an IR source. The system further comprises means for generating a tracking output image, the output image comprising a first region of light polarized in a first polarization direction and substantially conforming to the facial image fragment, and a second region of light polarized in a second polarization direction orthogonal to the first polarization direction. Two displays produce complementary stereoimages, and the tracking output image is directed through each of the displays so as to illuminate them in accordance with the polarizations of the tracking output image. The facial image fragment is focused onto a corresponding portion of the viewer""s face through the first display. Each of a pair of polarizers is interposed between the tracking output image and each of the displays; one of the polarizers is oriented in the first polarization direction and the other in the second polarization direction. This arrangement presents illumination from the appropriate display to the first eye of the viewer and illumination from the other display to the viewer""s other eye. Naturally, as with all embodiments of the invention, the stereoimages may represent a single, still stereopair or may instead change rapidly over time to convey movement.
In a second embodiment of the invention, polarization is used to segregate stereoscopic images for presentation on a display screen. In accordance with this embodiment, the stereoimages are combined through a projection lens onto a rear-projection display that directs each image component to the proper eye of the viewer.
In accordance with the second embodiment, a tracking system acquires a facial image fragment of the viewer, the facial image fragment including a first one of the viewer""s eyes. First and second complementary stereoimages are polarized in first and second polarization directions, respectively, and then combined (e.g., by a beamsplitter) into a composite image. A projection lens system projects the composite image onto a viewable display. Before the projected composite image reaches the display, however, it passes through means for separating the first and second images from the projected composite image. The image projection and/or display are controlled such that the viewer""s first eye receives light only from the first stereoimage and the viewer""s other eye receives light only from the second stereoimage.
In a preferred implementation of this embodiment, the means for separating the first and second images generates a tracking polarization pattern. This pattern has a first region substantially correlated to the facial image fragment (i.e., having the same general contour as the facial image fragment, although at a different scale) and a second region separate from the first region (e.g., the remainder of the pattern), and the pattern operates to alter the polarization state of the composite image. In particular, the pattern rotates the first or second region to a first polarization direction but rotates the other region to a second polarization direction. The altered light then passes through an output polarizer (disposed between the tracking polarization pattern and the display) that passes only that portion of the pattern-modified composite image polarized in the first or second direction. Which direction the polarizer passes depends on the polarizations initially applied to the two stereoimages, as discussed below.
The image exiting the polarizer reaches the display, which may be, for example, a lens functioning as a projection screen. The lens is situated so as to direct that portion of the composite image which has passed through the first region of the pattern onto the region of the viewer""s face from which the first region was derived. Suppose, for example, that the facial image fragment is drawn from the left side of the viewer""s face, the first region of the pattern (which is defined by this fragment) rotates the polarization of the incoming light by 90xc2x0, and that the output polarizer is oriented vertically. Assuming the left stereoimage is initially polarized horizontally, the viewer""s left eye will receive only light originating with the left stereoimage. This is because light directed toward the viewer""s left eye has been rotated 90xc2x0 by the first region of the pattern; the horizontally polarized light from the left stereoimage now passes through the vertical output polarizer, while the light from the right stereoimage, which was polarized vertically but has now been rotated 90xc2x0, is absorbed by the output polarizer. The opposite effect occurs with respect to the remainder of the composite image, which is directed toward the viewer""s right eye. Since this light is unrotated, only the vertical componentxe2x80x94originating with the right stereoimagexe2x80x94can pass through the output polarizer. As a result, the proper image is continuously directed toward the proper eye of the viewer.
A third embodiment of the invention provides a projection system that is polarization-independent. In accordance with this embodiment, two LCDs are used as xe2x80x9clight valvesxe2x80x9d to restrict the output of separate projection subsystems, each of which projects one image of a stereopair onto a beamsplitter that combines the images. The combined images are viewed through a display such as a projection screen (as in the second embodiment). Each LCD passes a small region of light corresponding to a viewing zone. The size and position of each region, combined with the geometry and optics of the display, operate to ensure that each of the viewer""s eyes receives light only from the proper stereoimage. It should be stressed that, as used herein, the term xe2x80x9clight valvexe2x80x9d connotes either variable illumination restriction or generation. For example, in the restrictive case described above, a source of illumination is positioned behind the light valve, which allows the light to pass only through a defined region. In the variable-illumination case, the light valve itself generates the desired region of illumination.
In a fourth embodiment, a single image source presents left-eye and right-eye images in rapid succession, and a light valve, in cooperation with the image source, synchronously presents the alternating images to the proper eye of the viewer (whose position is tracked). So long as the device performs this cycle of alternating images and light-valve regions at a high enough frequency, the viewer will see a proper three-dimensional image through the single image display.
One implementation of this embodiment includes an image source that successively displays left and right stereoimages, a light valve for providing a controllably sized and positioned region of light, and a focusing system for passing light from the light valve through the image source so as to present an image. Based on the tracked position of a viewer, a controller actuates the light valve so as to alternately define, in synchrony with display by the image source of left and right stereoimages, a pair of light regions comprising a left-eye region through which an image from the image source will appear in a left view zone visible to the viewer""s left eye, and a right-eye region through which an image from the image source will appear in a right view zone visible to the viewer""s right eye.
In a second implementation, the image appears on a projection screen or similar display device. In accordance with this implementation, light from the image source is directed through a projection lens onto the display screen, which may itself be a lens or lens system. The light valve is disposed between the image source and the display screen, at or near the projection means, which directs toward the left view zone light provided by the left-eye region of the light valve and directs toward the right view zone light provided by the right-eye region.
The invention also embodies methods relating to the above-described systems and various components thereof.